Active Role of Self-Sustained Neural Activity on Sensory Input Processing: A Minimal Theoretical Model.

A growing body of work has demonstrated the importance of ongoing oscillatory neural activity in sensory processing and the generation of sensorimotor behaviors. It has been shown, for several different brain areas, that sensory-evoked neural oscillations are generated from the modulation by sensory inputs of inherent self-sustained neural activity (SSA). This letter contributes to that strand of research by introducing a methodology to investigate how much of the sensory-evoked oscillatory activity is generated by SSA and how much is generated by sensory inputs within the context of sensorimotor behavior in a computational model. We develop an abstract model consisting of a network of three Kuramoto oscillators controlling the behavior of a simulated agent performing a categorical perception task. The effects of sensory inputs and SSAs on sensory-evoked oscillations are quantified by the cross product of velocity vectors in the phase space of the network under different conditions (disconnected without input, connected without input, and connected with input). We found that while the agent is carrying out the task, sensory-evoked activity is predominantly generated by SSA (93.10%) with much less influence from sensory inputs (6.90%). Furthermore, the influence of sensory inputs can be reduced by 10.4% (from 6.90% to 6.18%) with a decay in the agent's performance of only 2%. A dynamical analysis shows how sensory-evoked oscillations are generated from a dynamic coupling between the level of sensitivity of the network and the intensity of the input signals. This work may suggest interesting directions for neurophysiological experiments investigating how self-sustained neural activity influences sensory input processing, and ultimately affects behavior.

Evolved Transistor Array Robot Controllers.

For the first time, a field programmable transistor array (FPTA) was used to evolve robot control circuits directly in analog hardware. Controllers were successfully incrementally evolved for a physical robot engaged in a series of visually guided behaviours, including finding a target in a complex environment where the goal was hidden from most locations. Circuits for recognising spoken commands were also evolved and these were used in conjunction with the controllers to enable voice control of the robot, triggering behavioural switching. Poor quality visual sensors were deliberately used to test the ability of evolved analog circuits to deal with noisy uncertain data in realtime. Visual features were coevolved with the controllers to automatically achieve dimensionality reduction and feature extraction and selection in an integrated way. An efficient new method was developed for simulating the robot in its visual environment. This allowed controllers to be evaluated in a simulation connected to the FPTA. The controllers then transferred seamlessly to the real world. The circuit replication issue was also addressed in experiments where circuits were evolved to be able to function correctly in multiple areas of the FPTA. A methodology was developed to analyse the evolved circuits which provided insights into their operation. Comparative experiments demonstrated the superior evolvability of the transistor array medium.

Encoding Temporal Regularities and Information Copying in Hippocampal Circuits.

Discriminating, extracting and encoding temporal regularities is a critical requirement in the brain, relevant to sensory-motor processing and learning. However, the cellular mechanisms responsible remain enigmatic; for example, whether such abilities require specific, elaborately organized neural networks or arise from more fundamental, inherent properties of neurons. Here, using multi-electrode array technology, and focusing on interval learning, we demonstrate that sparse reconstituted rat hippocampal neural circuits are intrinsically capable of encoding and storing sub-second-order time intervals for over an hour timescale, represented in changes in the spatial-temporal architecture of firing relationships among populations of neurons. This learning is accompanied by increases in mutual information and transfer entropy, formal measures related to information storage and flow. Moreover, temporal relationships derived from previously trained circuits can act as templates for copying intervals into untrained networks, suggesting the possibility of circuit-to-circuit information transfer. Our findings illustrate that dynamic encoding and stable copying of temporal relationships are fundamental properties of simple in vitro networks, with general significance for understanding elemental principles of information processing, storage and replication.

Embodied neuromechanical chaos through homeostatic regulation.

In this paper, we present detailed analyses of the dynamics of a number of embodied neuromechanical systems of a class that has been shown to efficiently exploit chaos in the development and learning of motor behaviors for bodies of arbitrary morphology. This class of systems has been successfully used in robotics, as well as to model biological systems. At the heart of these systems are neural central pattern generating (CPG) units connected to actuators which return proprioceptive information via an adaptive homeostatic mechanism. Detailed dynamical analyses of example systems, using high resolution largest Lyapunov exponent maps, demonstrate the existence of chaotic regimes within a particular region of parameter space, as well as the striking similarity of the maps for systems of varying size. Thanks to the homeostatic sensory mechanisms, any single CPG "views" the whole of the rest of the system as if it was another CPG in a two coupled system, allowing a scale invariant conceptualization of such embodied neuromechanical systems. The analysis reveals chaos at all levels of the systems; the entire brain-body-environment system exhibits chaotic dynamics which can be exploited to power an exploration of possible motor behaviors. The crucial influence of the adaptive homeostatic mechanisms on the system dynamics is examined in detail, revealing chaotic behavior characterized by mixed mode oscillations (MMOs). An analysis of the mechanism of the MMO concludes that they stems from dynamic Hopf bifurcation, where a number of slow variables act as "moving" bifurcation parameters for the remaining part of the system.

Unsupervised Learning in an Ensemble of Spiking Neural Networks Mediated by ITDP.

We propose a biologically plausible architecture for unsupervised ensemble learning in a population of spiking neural network classifiers. A mixture of experts type organisation is shown to be effective, with the individual classifier outputs combined via a gating network whose operation is driven by input timing dependent plasticity (ITDP). The ITDP gating mechanism is based on recent experimental findings. An abstract, analytically tractable model of the ITDP driven ensemble architecture is derived from a logical model based on the probabilities of neural firing events. A detailed analysis of this model provides insights that allow it to be extended into a full, biologically plausible, computational implementation of the architecture which is demonstrated on a visual classification task. The extended model makes use of a style of spiking network, first introduced as a model of cortical microcircuits, that is capable of Bayesian inference, effectively performing expectation maximization. The unsupervised ensemble learning mechanism, based around such spiking expectation maximization (SEM) networks whose combined outputs are mediated by ITDP, is shown to perform the visual classification task well and to generalize to unseen data. The combined ensemble performance is significantly better than that of the individual classifiers, validating the ensemble architecture and learning mechanisms. The properties of the full model are analysed in the light of extensive experiments with the classification task, including an investigation into the influence of different input feature selection schemes and a comparison with a hierarchical STDP based ensemble architecture.

Neuronal assembly dynamics in supervised and unsupervised learning scenarios.

The dynamic formation of groups of neurons--neuronal assemblies--is believed to mediate cognitive phenomena at many levels, but their detailed operation and mechanisms of interaction are still to be uncovered. One hypothesis suggests that synchronized oscillations underpin their formation and functioning, with a focus on the temporal structure of neuronal signals. In this context, we investigate neuronal assembly dynamics in two complementary scenarios: the first, a supervised spike pattern classification task, in which noisy variations of a collection of spikes have to be correctly labeled; the second, an unsupervised, minimally cognitive evolutionary robotics tasks, in which an evolved agent has to cope with multiple, possibly conflicting, objectives. In both cases, the more traditional dynamical analysis of the system's variables is paired with information-theoretic techniques in order to get a broader picture of the ongoing interactions with and within the network. The neural network model is inspired by the Kuramoto model of coupled phase oscillators and allows one to fine-tune the network synchronization dynamics and assembly configuration. The experiments explore the computational power, redundancy, and generalization capability of neuronal circuits, demonstrating that performance depends nonlinearly on the number of assemblies and neurons in the network and showing that the framework can be exploited to generate minimally cognitive behaviors, with dynamic assembly formation accounting for varying degrees of stimuli modulation of the sensorimotor interactions.

Evolution of associative learning in chemical networks.

Organisms that can learn about their environment and modify their behaviour appropriately during their lifetime are more likely to survive and reproduce than organisms that do not. While associative learning - the ability to detect correlated features of the environment - has been studied extensively in nervous systems, where the underlying mechanisms are reasonably well understood, mechanisms within single cells that could allow associative learning have received little attention. Here, using in silico evolution of chemical networks, we show that there exists a diversity of remarkably simple and plausible chemical solutions to the associative learning problem, the simplest of which uses only one core chemical reaction. We then asked to what extent a linear combination of chemical concentrations in the network could approximate the ideal Bayesian posterior of an environment given the stimulus history so far? This Bayesian analysis revealed the 'memory traces' of the chemical network. The implication of this paper is that there is little reason to believe that a lack of suitable phenotypic variation would prevent associative learning from evolving in cell signalling, metabolic, gene regulatory, or a mixture of these networks in cells.

Synchronisation effects on the behavioural performance and information dynamics of a simulated minimally cognitive robotic agent.

Oscillatory activity is ubiquitous in nervous systems, with solid evidence that synchronisation mechanisms underpin cognitive processes. Nevertheless, its informational content and relationship with behaviour are still to be fully understood. In addition, cognitive systems cannot be properly appreciated without taking into account brain-body- environment interactions. In this paper, we developed a model based on the Kuramoto Model of coupled phase oscillators to explore the role of neural synchronisation in the performance of a simulated robotic agent in two different minimally cognitive tasks. We show that there is a statistically significant difference in performance and evolvability depending on the synchronisation regime of the network. In both tasks, a combination of information flow and dynamical analyses show that networks with a definite, but not too strong, propensity for synchronisation are more able to reconfigure, to organise themselves functionally and to adapt to different behavioural conditions. The results highlight the asymmetry of information flow and its behavioural correspondence. Importantly, it also shows that neural synchronisation dynamics, when suitably flexible and reconfigurable, can generate minimally cognitive embodied behaviour.

Selectionist and evolutionary approaches to brain function: a critical appraisal.

We consider approaches to brain dynamics and function that have been claimed to be Darwinian. These include Edelman's theory of neuronal group selection, Changeux's theory of synaptic selection and selective stabilization of pre-representations, Seung's Darwinian synapse, Loewenstein's synaptic melioration, Adam's selfish synapse, and Calvin's replicating activity patterns. Except for the last two, the proposed mechanisms are selectionist but not truly Darwinian, because no replicators with information transfer to copies and hereditary variation can be identified in them. All of them fit, however, a generalized selectionist framework conforming to the picture of Price's covariance formulation, which deliberately was not specific even to selection in biology, and therefore does not imply an algorithmic picture of biological evolution. Bayesian models and reinforcement learning are formally in agreement with selection dynamics. A classification of search algorithms is shown to include Darwinian replicators (evolutionary units with multiplication, heredity, and variability) as the most powerful mechanism for search in a sparsely occupied search space. Examples are given of cases where parallel competitive search with information transfer among the units is more efficient than search without information transfer between units. Finally, we review our recent attempts to construct and analyze simple models of true Darwinian evolutionary units in the brain in terms of connectivity and activity copying of neuronal groups. Although none of the proposed neuronal replicators include miraculous mechanisms, their identification remains a challenge but also a great promise.

Chaotic exploration and learning of locomotion behaviors.

We present a general and fully dynamic neural system, which exploits intrinsic chaotic dynamics, for the real-time goal-directed exploration and learning of the possible locomotion patterns of an articulated robot of an arbitrary morphology in an unknown environment. The controller is modeled as a network of neural oscillators that are initially coupled only through physical embodiment, and goal-directed exploration of coordinated motor patterns is achieved by chaotic search using adaptive bifurcation. The phase space of the indirectly coupled neural-body-environment system contains multiple transient or permanent self-organized dynamics, each of which is a candidate for a locomotion behavior. The adaptive bifurcation enables the system orbit to wander through various phase-coordinated states, using its intrinsic chaotic dynamics as a driving force, and stabilizes on to one of the states matching the given goal criteria. In order to improve the sustainability of useful transient patterns, sensory homeostasis has been introduced, which results in an increased diversity of motor outputs, thus achieving multiscale exploration. A rhythmic pattern discovered by this process is memorized and sustained by changing the wiring between initially disconnected oscillators using an adaptive synchronization method. Our results show that the novel neurorobotic system is able to create and learn multiple locomotion behaviors for a wide range of body configurations and physical environments and can readapt in realtime after sustaining damage.

Evolvable physical self-replicators.

Building an evolvable physical self-replicating machine is a grand challenge. The main problem is that the device must be capable of hereditary variation, that is, replicating in many configurations-configurations into which it enters unpredictably by mutation. Template replication is the solution found by nature. A scalable device must also be capable of miniaturization, and so have few or no moving and electronic parts. Here a significant step toward this goal is presented in the form of a physical template replicator made from small plastic pieces containing embedded magnets that float on an air-hockey-type table and undergo stochastic motion. Our units replicate by a process analogous to the replication of DNA, except without the involvement of enzymes. Building a physical rather than a computational model forces us to confront several problems that have analogues on the nano scale. In particular, replication must be maintained by preventing side reactions such as spontaneous ligation, cyclization, product inhibition, and elongation at staggered ends. The last of these results in ever-lengthening sequences in a process known as the elongation catastrophe. The extreme specificity of structure required by the monomers is indirect evidence that some kind of natural selection took place prior to the existence of nucleotide analogues during the origin of life.

Evolvable neuronal paths: a novel basis for information and search in the brain.

We propose a previously unrecognized kind of informational entity in the brain that is capable of acting as the basis for unlimited hereditary variation in neuronal networks. This unit is a path of activity through a network of neurons, analogous to a path taken through a hidden Markov model. To prove in principle the capabilities of this new kind of informational substrate, we show how a population of paths can be used as the hereditary material for a neuronally implemented genetic algorithm, (the swiss-army knife of black-box optimization techniques) which we have proposed elsewhere could operate at somatic timescales in the brain. We compare this to the same genetic algorithm that uses a standard 'genetic' informational substrate, i.e. non-overlapping discrete genotypes, on a range of optimization problems. A path evolution algorithm (PEA) is defined as any algorithm that implements natural selection of paths in a network substrate. A PEA is a previously unrecognized type of natural selection that is well suited for implementation by biological neuronal networks with structural plasticity. The important similarities and differences between a standard genetic algorithm and a PEA are considered. Whilst most experiments are conducted on an abstract network model, at the conclusion of the paper a slightly more realistic neuronal implementation of a PEA is outlined based on Izhikevich spiking neurons. Finally, experimental predictions are made for the identification of such informational paths in the brain.

Spike-timing dependent plasticity and the cognitive map.

Since the discovery of place cells - single pyramidal neurons that encode spatial location - it has been hypothesized that the hippocampus may act as a cognitive map of known environments. This putative function has been extensively modeled using auto-associative networks, which utilize rate-coded synaptic plasticity rules in order to generate strong bi-directional connections between concurrently active place cells that encode for neighboring place fields. However, empirical studies using hippocampal cultures have demonstrated that the magnitude and direction of changes in synaptic strength can also be dictated by the relative timing of pre- and post-synaptic firing according to a spike-timing dependent plasticity (STDP) rule. Furthermore, electrophysiology studies have identified persistent "theta-coded" temporal correlations in place cell activity in vivo, characterized by phase precession of firing as the corresponding place field is traversed. It is not yet clear if STDP and theta-coded neural dynamics are compatible with cognitive map theory and previous rate-coded models of spatial learning in the hippocampus. Here, we demonstrate that an STDP rule based on empirical data obtained from the hippocampus can mediate rate-coded Hebbian learning when pre- and post-synaptic activity is stochastic and has no persistent sequence bias. We subsequently demonstrate that a spiking recurrent neural network that utilizes this STDP rule, alongside theta-coded neural activity, allows the rapid development of a cognitive map during directed or random exploration of an environment of overlapping place fields. Hence, we establish that STDP and phase precession are compatible with rate-coded models of cognitive map development.

Dual coding with STDP in a spiking recurrent neural network model of the hippocampus.

The firing rate of single neurons in the mammalian hippocampus has been demonstrated to encode for a range of spatial and non-spatial stimuli. It has also been demonstrated that phase of firing, with respect to the theta oscillation that dominates the hippocampal EEG during stereotype learning behaviour, correlates with an animal's spatial location. These findings have led to the hypothesis that the hippocampus operates using a dual (rate and temporal) coding system. To investigate the phenomenon of dual coding in the hippocampus, we examine a spiking recurrent network model with theta coded neural dynamics and an STDP rule that mediates rate-coded Hebbian learning when pre- and post-synaptic firing is stochastic. We demonstrate that this plasticity rule can generate both symmetric and asymmetric connections between neurons that fire at concurrent or successive theta phase, respectively, and subsequently produce both pattern completion and sequence prediction from partial cues. This unifies previously disparate auto- and hetero-associative network models of hippocampal function and provides them with a firmer basis in modern neurobiology. Furthermore, the encoding and reactivation of activity in mutually exciting Hebbian cell assemblies demonstrated here is believed to represent a fundamental mechanism of cognitive processing in the brain.

Reconciling the STDP and BCM models of synaptic plasticity in a spiking recurrent neural network.

Rate-coded Hebbian learning, as characterized by the BCM formulation, is an established computational model of synaptic plasticity. Recently it has been demonstrated that changes in the strength of synapses in vivo can also depend explicitly on the relative timing of pre- and postsynaptic firing. Computational modeling of this spike-timing-dependent plasticity (STDP) has demonstrated that it can provide inherent stability or competition based on local synaptic variables. However, it has also been demonstrated that these properties rely on synaptic weights being either depressed or unchanged by an increase in mean stochastic firing rates, which directly contradicts empirical data. Several analytical studies have addressed this apparent dichotomy and identified conditions under which distinct and disparate STDP rules can be reconciled with rate-coded Hebbian learning. The aim of this research is to verify, unify, and expand on these previous findings by manipulating each element of a standard computational STDP model in turn. This allows us to identify the conditions under which this plasticity rule can replicate experimental data obtained using both rate and temporal stimulation protocols in a spiking recurrent neural network. Our results describe how the relative scale of mean synaptic weights and their dependence on stochastic pre- or postsynaptic firing rates can be manipulated by adjusting the exact profile of the asymmetric learning window and temporal restrictions on spike pair interactions respectively. These findings imply that previously disparate models of rate-coded autoassociative learning and temporally coded heteroassociative learning, mediated by symmetric and asymmetric connections respectively, can be implemented in a single network using a single plasticity rule. However, we also demonstrate that forms of STDP that can be reconciled with rate-coded Hebbian learning do not generate inherent synaptic competition, and thus some additional mechanism is required to guarantee long-term input-output selectivity.

In silico biology.

Rather than studying existent living systems, we can increasingly produce computer models that capture the salient aspects of life. This provides us with unprecedented opportunities to examine, manipulate, and explore biological phenomena, allowing us to investigate some of the deepest issues in biology.

The evolution of reaction-diffusion controllers for minimally cognitive agents.

This article describes work carried out to investigate whether a classic reaction-diffusion (RD) system could be used to control a minimally cognitive animat. The RD system chosen was that first described by Gray and Scott, and the minimally cognitive behaviors were those used by Beer et al. involving the fixation and discrimination of diamond and circle shapes by a whiskered animat. A further task was added, which required the RD controllers to maintain and use a chemical memory. The parameters of these controllers were evolved using an evolutionary, or genetic, algorithm.

Flexible couplings: diffusing neuromodulators and adaptive robotics.

Recent years have seen the discovery of freely diffusing gaseous neurotransmitters, such as nitric oxide (NO), in biological nervous systems. A type of artificial neural network (ANN) inspired by such gaseous signaling, the GasNet, has previously been shown to be more evolvable than traditional ANNs when used as an artificial nervous system in an evolutionary robotics setting, where evolvability means consistent speed to very good solutions--here, appropriate sensorimotor behavior-generating systems. We present two new versions of the GasNet, which take further inspiration from the properties of neuronal gaseous signaling. The plexus model is inspired by the extraordinary NO-producing cortical plexus structure of neural fibers and the properties of the diffusing NO signal it generates. The receptor model is inspired by the mediating action of eurotransmitter receptors. Both models are shown to significantly further improve evolvability. We describe a series of analyses suggesting that the reasons for the increase in evolvability are related to the flexible loose coupling of distinct signaling mechanisms, one "chemical" and one "electrical."

Evolving controllers for a homogeneous system of physical robots: structured cooperation with minimal sensors.

We report on recent work in which we employed artificial evolution to design neural network controllers for small, homogeneous teams of mobile autonomous robots. The robots were evolved to perform a formation-movement task from random starting positions, equipped only with infrared sensors. The dual constraints of homogeneity and minimal sensors make this a non-trivial task. We describe the behaviour of a successful system in which robots adopt and maintain functionally distinct roles in order to achieve the task. We believe this to be the first example of the use of artificial evolution to design coordinated, cooperative behaviour for real robots.

Local evolvability of statistically neutral GasNet robot controllers.

In this paper we introduce and apply the concept of local evolvability to investigate the behaviour of populations during evolutionary search. We focus on the evolution of GasNet neural network controllers for a robotic visual discrimination problem, showing that the evolutionary process undergoes long neutral fitness epochs. We show that the local evolvability properties of the search space surrounding a group of statistically neutral solutions do vary across the course of an evolutionary run, especially during periods of population takeover. However, once takeover is complete there is no evidence for further increase in local evolvability across fitness epochs. We also see no evidence for the neutral evolution of increased solution robustness, but show that this may be due to the ability of evolutionary algorithms to focus search on volumes of the fitness landscape with above average robustness.